{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8ZelAXba3D05"
   },
   "source": [
    "![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/nlu/blob/master/examples/colab/healthcare/de_identification/DeIdentification_model_overview.ipynb)\n",
    "\n",
    "All the models avaiable are :\n",
    "\n",
    "| Language | nlp.load() reference                                         | Spark NLP Model reference          |\n",
    "| -------- | ------------------------------------------------------------ | ------------------------------------------------------------ |\n",
    "| English  | med_ner.deid | nerdl_deid          |\n",
    "| English  | [en.de_identify](https://nlp.johnsnowlabs.com/2019/06/04/deidentify_rb_en.html) | [deidentify_rb](https://nlp.johnsnowlabs.com/2019/06/04/deidentify_rb_en.html)           |\n",
    "| English  | de_identify.rules | deid_rules          |\n",
    "| English  | [de_identify.clinical](https://nlp.johnsnowlabs.com/2021/01/29/deidentify_enriched_clinical_en.html) | [deidentify_enriched_clinical](https://nlp.johnsnowlabs.com/2021/01/29/deidentify_enriched_clinical_en.html)           |\n",
    "| English  | [de_identify.large](https://nlp.johnsnowlabs.com/2020/08/04/deidentify_large_en.html) | [deidentify_large](https://nlp.johnsnowlabs.com/2020/08/04/deidentify_large_en.html)           |\n",
    "| English  | [de_identify.rb](https://nlp.johnsnowlabs.com/2019/06/04/deidentify_rb_en.html) | [deidentify_rb](https://nlp.johnsnowlabs.com/2019/06/04/deidentify_rb_en.html)           |\n",
    "| English  | de_identify.rb_no_regex | deidentify_rb_no_regex          |\n",
    "| English  | [resolve_chunk.athena_conditions](https://nlp.johnsnowlabs.com/2020/09/16/chunkresolve_athena_conditions_healthcare_en.html) | [chunkresolve_athena_conditions_healthcare](https://nlp.johnsnowlabs.com/2020/09/16/chunkresolve_athena_conditions_healthcare_en.html)           |\n",
    "| English  | [resolve_chunk.cpt_clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_cpt_clinical_en.html) | [chunkresolve_cpt_clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_cpt_clinical_en.html)           |\n",
    "| English  | [resolve_chunk.icd10cm.clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10cm_clinical_en.html) | [chunkresolve_icd10cm_clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10cm_clinical_en.html)           |\n",
    "| English  | [resolve_chunk.icd10cm.diseases_clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10cm_diseases_clinical_en.html) | [chunkresolve_icd10cm_diseases_clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10cm_diseases_clinical_en.html)           |\n",
    "| English  | resolve_chunk.icd10cm.hcc_clinical | chunkresolve_icd10cm_hcc_clinical          |\n",
    "| English  | resolve_chunk.icd10cm.hcc_healthcare | chunkresolve_icd10cm_hcc_healthcare          |\n",
    "| English  | [resolve_chunk.icd10cm.injuries](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10cm_injuries_clinical_en.html) | [chunkresolve_icd10cm_injuries_clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10cm_injuries_clinical_en.html)           |\n",
    "| English  | [resolve_chunk.icd10cm.musculoskeletal](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10cm_musculoskeletal_clinical_en.html) | [chunkresolve_icd10cm_musculoskeletal_clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10cm_musculoskeletal_clinical_en.html)           |\n",
    "| English  | [resolve_chunk.icd10cm.neoplasms](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10cm_neoplasms_clinical_en.html) | [chunkresolve_icd10cm_neoplasms_clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10cm_neoplasms_clinical_en.html)           |\n",
    "| English  | [resolve_chunk.icd10cm.poison](https://nlp.johnsnowlabs.com/2020/04/28/chunkresolve_icd10cm_poison_ext_clinical_en.html) | [chunkresolve_icd10cm_poison_ext_clinical](https://nlp.johnsnowlabs.com/2020/04/28/chunkresolve_icd10cm_poison_ext_clinical_en.html)           |\n",
    "| English  | [resolve_chunk.icd10cm.puerile](https://nlp.johnsnowlabs.com/2020/04/28/chunkresolve_icd10cm_puerile_clinical_en.html) | [chunkresolve_icd10cm_puerile_clinical](https://nlp.johnsnowlabs.com/2020/04/28/chunkresolve_icd10cm_puerile_clinical_en.html)           |\n",
    "| English  | resolve_chunk.icd10pcs.clinical | chunkresolve_icd10pcs_clinical          |\n",
    "| English  | [resolve_chunk.icdo.clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10pcs_clinical_en.html) | [chunkresolve_icdo_clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_icd10pcs_clinical_en.html)           |\n",
    "| English  | [resolve_chunk.loinc](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_loinc_clinical_en.html) | [chunkresolve_loinc_clinical](https://nlp.johnsnowlabs.com/2021/04/02/chunkresolve_loinc_clinical_en.html)           |\n",
    "| English  | [resolve_chunk.rxnorm.cd](https://nlp.johnsnowlabs.com/2020/07/27/chunkresolve_rxnorm_cd_clinical_en.html) | [chunkresolve_rxnorm_cd_clinical](https://nlp.johnsnowlabs.com/2020/07/27/chunkresolve_rxnorm_cd_clinical_en.html)           |\n",
    "| English  | resolve_chunk.rxnorm.in | chunkresolve_rxnorm_in_clinical          |\n",
    "| English  | resolve_chunk.rxnorm.in_healthcare | chunkresolve_rxnorm_in_healthcare          |\n",
    "| English  | [resolve_chunk.rxnorm.sbd](https://nlp.johnsnowlabs.com/2020/07/27/chunkresolve_rxnorm_sbd_clinical_en.html) | [chunkresolve_rxnorm_sbd_clinical](https://nlp.johnsnowlabs.com/2020/07/27/chunkresolve_rxnorm_sbd_clinical_en.html)           |\n",
    "| English  | [resolve_chunk.rxnorm.scd](https://nlp.johnsnowlabs.com/2020/07/27/chunkresolve_rxnorm_scd_clinical_en.html) | [chunkresolve_rxnorm_scd_clinical](https://nlp.johnsnowlabs.com/2020/07/27/chunkresolve_rxnorm_scd_clinical_en.html)           |\n",
    "| English  | resolve_chunk.rxnorm.scdc | chunkresolve_rxnorm_scdc_clinical          |\n",
    "| English  | resolve_chunk.rxnorm.scdc_healthcare | chunkresolve_rxnorm_scdc_healthcare          |\n",
    "| English  | [resolve_chunk.rxnorm.xsmall.clinical](https://nlp.johnsnowlabs.com/2020/06/24/chunkresolve_rxnorm_xsmall_clinical_en.html) | [chunkresolve_rxnorm_xsmall_clinical](https://nlp.johnsnowlabs.com/2020/06/24/chunkresolve_rxnorm_xsmall_clinical_en.html)           |\n",
    "| English  | [resolve_chunk.snomed.findings](https://nlp.johnsnowlabs.com/2020/06/20/chunkresolve_snomed_findings_clinical_en.html) | [chunkresolve_snomed_findings_clinical](https://nlp.johnsnowlabs.com/2020/06/20/chunkresolve_snomed_findings_clinical_en.html)           |\n",
    "| English  | classify.icd10.clinical | classifier_icd10cm_hcc_clinical          |\n",
    "| English  | classify.icd10.healthcare | classifier_icd10cm_hcc_healthcare          |\n",
    "| English  | [classify.ade.biobert](https://nlp.johnsnowlabs.com/2021/01/21/classifierdl_ade_biobert_en.html) | [classifierdl_ade_biobert](https://nlp.johnsnowlabs.com/2021/01/21/classifierdl_ade_biobert_en.html)           |\n",
    "| English  | [classify.ade.clinical](https://nlp.johnsnowlabs.com/2021/01/21/classifierdl_ade_clinicalbert_en.html) | [classifierdl_ade_clinicalbert](https://nlp.johnsnowlabs.com/2021/01/21/classifierdl_ade_clinicalbert_en.html)           |\n",
    "| English  | [classify.ade.conversational](https://nlp.johnsnowlabs.com/2021/01/21/classifierdl_ade_conversational_biobert_en.html) | [classifierdl_ade_conversational_biobert](https://nlp.johnsnowlabs.com/2021/01/21/classifierdl_ade_conversational_biobert_en.html)           |\n",
    "| English  | [classify.gender.biobert](https://nlp.johnsnowlabs.com/2021/01/21/classifierdl_gender_biobert_en.html) | [classifierdl_gender_biobert](https://nlp.johnsnowlabs.com/2021/01/21/classifierdl_gender_biobert_en.html)           |\n",
    "| English  | [classify.gender.sbert](https://nlp.johnsnowlabs.com/2021/01/21/classifierdl_gender_sbert_en.html) | [classifierdl_gender_sbert](https://nlp.johnsnowlabs.com/2021/01/21/classifierdl_gender_sbert_en.html)           |\n",
    "| English  | classify.pico | classifierdl_pico_biobert          |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "0f0EjD1DBXxU"
   },
   "outputs": [],
   "source": [
    "from johnsnowlabs import nlp\n",
    "\n",
    "# After uploading your license run this to install all licensed Python Wheels and pre-download Jars the Spark Session JVM\n",
    "nlp.install()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IvDy7_ddfrfU"
   },
   "source": [
    "#### [Deidentify RB](https://nlp.johnsnowlabs.com/2019/06/04/deidentify_rb_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 667
    },
    "id": "GclZvHWzZuZp",
    "outputId": "fae4b4c4-5069-42f4-a49a-532effbdab0f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_wikiner_glove_840B_300 download started this may take some time.\n",
      "Approximate size to download 14.8 MB\n",
      "[OK!]\n",
      "deidentify_rb download started this may take some time.\n",
      "[OK!]\n",
      "glove_840B_300 download started this may take some time.\n",
      "Approximate size to download 2.3 GB\n",
      "[OK!]\n",
      "sentence_detector_dl download started this may take some time.\n",
      "Approximate size to download 354.6 KB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-452a890a-cc05-4fdc-af04-25095a84e0fb\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>de_identified</th>\n",
       "      <th>entities_wikiner_glove_840B_300</th>\n",
       "      <th>entities_wikiner_glove_840B_300_class</th>\n",
       "      <th>entities_wikiner_glove_840B_300_confidence</th>\n",
       "      <th>entities_wikiner_glove_840B_300_origin_chunk</th>\n",
       "      <th>entities_wikiner_glove_840B_300_origin_sentence</th>\n",
       "      <th>sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . &lt;ORG&gt; date : &lt;DATE&gt; , &lt;PER&gt; , &lt;MISC&gt; , &lt;PE...</td>\n",
       "      <td>[Record, David Hale, M.D . , Name : Hendrickso...</td>\n",
       "      <td>[ORG, PER, MISC, PER, PER, MISC, LOC, LOC]</td>\n",
       "      <td>[0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 1, 2, 2]</td>\n",
       "      <td>A . Record date : 2093-01-13 , David Hale , M....</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . &lt;ORG&gt; date : &lt;DATE&gt; , &lt;PER&gt; , &lt;MISC&gt; , &lt;PE...</td>\n",
       "      <td>[Record, David Hale, M.D . , Name : Hendrickso...</td>\n",
       "      <td>[ORG, PER, MISC, PER, PER, MISC, LOC, LOC]</td>\n",
       "      <td>[0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 1, 2, 2]</td>\n",
       "      <td>Oliveira , 25 years-old , Record date : 2079-1...</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . &lt;ORG&gt; date : &lt;DATE&gt; , &lt;PER&gt; , &lt;MISC&gt; , &lt;PE...</td>\n",
       "      <td>[Record, David Hale, M.D . , Name : Hendrickso...</td>\n",
       "      <td>[ORG, PER, MISC, PER, PER, MISC, LOC, LOC]</td>\n",
       "      <td>[0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 1, 2, 2]</td>\n",
       "      <td>Cocke County Baptist Hospital . 0295 Keats St...</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-452a890a-cc05-4fdc-af04-25095a84e0fb')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-452a890a-cc05-4fdc-af04-25095a84e0fb button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-452a890a-cc05-4fdc-af04-25095a84e0fb');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-b7540932-54ad-4c66-84d4-79702ad25299\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-b7540932-54ad-4c66-84d4-79702ad25299')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-b7540932-54ad-4c66-84d4-79702ad25299 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                       de_identified  \\\n",
       "0  A . <ORG> date : <DATE> , <PER> , <MISC> , <PE...   \n",
       "0  A . <ORG> date : <DATE> , <PER> , <MISC> , <PE...   \n",
       "0  A . <ORG> date : <DATE> , <PER> , <MISC> , <PE...   \n",
       "\n",
       "                     entities_wikiner_glove_840B_300  \\\n",
       "0  [Record, David Hale, M.D . , Name : Hendrickso...   \n",
       "0  [Record, David Hale, M.D . , Name : Hendrickso...   \n",
       "0  [Record, David Hale, M.D . , Name : Hendrickso...   \n",
       "\n",
       "        entities_wikiner_glove_840B_300_class  \\\n",
       "0  [ORG, PER, MISC, PER, PER, MISC, LOC, LOC]   \n",
       "0  [ORG, PER, MISC, PER, PER, MISC, LOC, LOC]   \n",
       "0  [ORG, PER, MISC, PER, PER, MISC, LOC, LOC]   \n",
       "\n",
       "          entities_wikiner_glove_840B_300_confidence  \\\n",
       "0  [0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...   \n",
       "0  [0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...   \n",
       "0  [0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...   \n",
       "\n",
       "  entities_wikiner_glove_840B_300_origin_chunk  \\\n",
       "0                     [0, 1, 2, 3, 4, 5, 6, 7]   \n",
       "0                     [0, 1, 2, 3, 4, 5, 6, 7]   \n",
       "0                     [0, 1, 2, 3, 4, 5, 6, 7]   \n",
       "\n",
       "  entities_wikiner_glove_840B_300_origin_sentence  \\\n",
       "0                        [0, 0, 0, 0, 1, 1, 2, 2]   \n",
       "0                        [0, 0, 0, 0, 1, 1, 2, 2]   \n",
       "0                        [0, 0, 0, 0, 1, 1, 2, 2]   \n",
       "\n",
       "                                            sentence  \\\n",
       "0  A . Record date : 2093-01-13 , David Hale , M....   \n",
       "0  Oliveira , 25 years-old , Record date : 2079-1...   \n",
       "0   Cocke County Baptist Hospital . 0295 Keats St...   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = '''A . Record date : 2093-01-13 , David Hale , M.D . , Name : Hendrickson , Ora MR . # 7194334 Date : 01/13/93 PCP : Oliveira , 25 years-old , Record date : 2079-11-09 . Cocke County Baptist Hospital . 0295 Keats Street'''\n",
    "nlp.load(\"med_ner.jsl.wip.clinical en.de_identify\").predict(data,output_level = 'sentence')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Jb4hB3IBgbRG"
   },
   "source": [
    "#### [Deidentify (Enriched)](https://nlp.johnsnowlabs.com/2021/01/29/deidentify_enriched_clinical_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 667
    },
    "id": "l6LQ32FxgbRS",
    "outputId": "6450438f-968b-4065-f67b-713593907a13"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_wikiner_glove_840B_300 download started this may take some time.\n",
      "Approximate size to download 14.8 MB\n",
      "[OK!]\n",
      "deidentify_enriched_clinical download started this may take some time.\n",
      "[OK!]\n",
      "glove_840B_300 download started this may take some time.\n",
      "Approximate size to download 2.3 GB\n",
      "[OK!]\n",
      "sentence_detector_dl download started this may take some time.\n",
      "Approximate size to download 354.6 KB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-53844a93-7479-4a4a-bc18-9143a2b7df7d\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>de_identified</th>\n",
       "      <th>entities_wikiner_glove_840B_300</th>\n",
       "      <th>entities_wikiner_glove_840B_300_class</th>\n",
       "      <th>entities_wikiner_glove_840B_300_confidence</th>\n",
       "      <th>entities_wikiner_glove_840B_300_origin_chunk</th>\n",
       "      <th>entities_wikiner_glove_840B_300_origin_sentence</th>\n",
       "      <th>sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . Eli Lilly and Company date : 2093-01-13 , ...</td>\n",
       "      <td>[Record, David Hale, M.D . , Name : Hendrickso...</td>\n",
       "      <td>[ORG, PER, MISC, PER, PER, MISC, LOC, LOC]</td>\n",
       "      <td>[0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 1, 2, 2]</td>\n",
       "      <td>A . Record date : 2093-01-13 , David Hale , M....</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . Eli Lilly and Company date : 2093-01-13 , ...</td>\n",
       "      <td>[Record, David Hale, M.D . , Name : Hendrickso...</td>\n",
       "      <td>[ORG, PER, MISC, PER, PER, MISC, LOC, LOC]</td>\n",
       "      <td>[0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 1, 2, 2]</td>\n",
       "      <td>Oliveira , 25 years-old , Record date : 2079-1...</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . Eli Lilly and Company date : 2093-01-13 , ...</td>\n",
       "      <td>[Record, David Hale, M.D . , Name : Hendrickso...</td>\n",
       "      <td>[ORG, PER, MISC, PER, PER, MISC, LOC, LOC]</td>\n",
       "      <td>[0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 1, 2, 2]</td>\n",
       "      <td>Cocke County Baptist Hospital . 0295 Keats St...</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-53844a93-7479-4a4a-bc18-9143a2b7df7d')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-53844a93-7479-4a4a-bc18-9143a2b7df7d button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-53844a93-7479-4a4a-bc18-9143a2b7df7d');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-a4d48dad-9407-4bd9-a310-2a6e8be6849a\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-a4d48dad-9407-4bd9-a310-2a6e8be6849a')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-a4d48dad-9407-4bd9-a310-2a6e8be6849a button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                       de_identified  \\\n",
       "0  A . Eli Lilly and Company date : 2093-01-13 , ...   \n",
       "0  A . Eli Lilly and Company date : 2093-01-13 , ...   \n",
       "0  A . Eli Lilly and Company date : 2093-01-13 , ...   \n",
       "\n",
       "                     entities_wikiner_glove_840B_300  \\\n",
       "0  [Record, David Hale, M.D . , Name : Hendrickso...   \n",
       "0  [Record, David Hale, M.D . , Name : Hendrickso...   \n",
       "0  [Record, David Hale, M.D . , Name : Hendrickso...   \n",
       "\n",
       "        entities_wikiner_glove_840B_300_class  \\\n",
       "0  [ORG, PER, MISC, PER, PER, MISC, LOC, LOC]   \n",
       "0  [ORG, PER, MISC, PER, PER, MISC, LOC, LOC]   \n",
       "0  [ORG, PER, MISC, PER, PER, MISC, LOC, LOC]   \n",
       "\n",
       "          entities_wikiner_glove_840B_300_confidence  \\\n",
       "0  [0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...   \n",
       "0  [0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...   \n",
       "0  [0.3596, 0.731, 0.64778334, 0.49190003, 0.8544...   \n",
       "\n",
       "  entities_wikiner_glove_840B_300_origin_chunk  \\\n",
       "0                     [0, 1, 2, 3, 4, 5, 6, 7]   \n",
       "0                     [0, 1, 2, 3, 4, 5, 6, 7]   \n",
       "0                     [0, 1, 2, 3, 4, 5, 6, 7]   \n",
       "\n",
       "  entities_wikiner_glove_840B_300_origin_sentence  \\\n",
       "0                        [0, 0, 0, 0, 1, 1, 2, 2]   \n",
       "0                        [0, 0, 0, 0, 1, 1, 2, 2]   \n",
       "0                        [0, 0, 0, 0, 1, 1, 2, 2]   \n",
       "\n",
       "                                            sentence  \\\n",
       "0  A . Record date : 2093-01-13 , David Hale , M....   \n",
       "0  Oliveira , 25 years-old , Record date : 2079-1...   \n",
       "0   Cocke County Baptist Hospital . 0295 Keats St...   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = '''A . Record date : 2093-01-13 , David Hale , M.D . , Name : Hendrickson , Ora MR . # 7194334 Date : 01/13/93 PCP : Oliveira , 25 years-old , Record date : 2079-11-09 . Cocke County Baptist Hospital . 0295 Keats Street'''\n",
    "nlp.load(\"med_ner.jsl.wip.clinical en.de_identify.clinical\").predict(data,output_level = 'sentence')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j0Sj05zHgrRn"
   },
   "source": [
    "#### [Deidentify PHI (Large)](https://nlp.johnsnowlabs.com/2020/08/04/deidentify_large_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "6JK2_IyLgrRo",
    "outputId": "435c3075-ab25-4ff5-8eeb-39ead1571c3b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_wikiner_glove_840B_300 download started this may take some time.\n",
      "Approximate size to download 14.8 MB\n",
      "[OK!]\n",
      "deidentify_large download started this may take some time.\n",
      "[OK!]\n",
      "glove_840B_300 download started this may take some time.\n",
      "Approximate size to download 2.3 GB\n",
      "[OK!]\n",
      "sentence_detector_dl download started this may take some time.\n",
      "Approximate size to download 354.6 KB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-fa7c6047-8416-4aec-986d-4bac3a740721\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>de_identified</th>\n",
       "      <th>entities_wikiner_glove_840B_300</th>\n",
       "      <th>entities_wikiner_glove_840B_300_class</th>\n",
       "      <th>entities_wikiner_glove_840B_300_confidence</th>\n",
       "      <th>entities_wikiner_glove_840B_300_origin_chunk</th>\n",
       "      <th>entities_wikiner_glove_840B_300_origin_sentence</th>\n",
       "      <th>sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;MISC&gt;, 25 month years-old , born in One Parkl...</td>\n",
       "      <td>[Patient AIQING, Beijing, The Johns Hopkins Ho...</td>\n",
       "      <td>[MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...</td>\n",
       "      <td>[0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 6, 7, 7, 8, 8]</td>\n",
       "      <td>Patient AIQING, 25 month years-old , born in B...</td>\n",
       "      <td>[[-0.4230000078678131, -0.027002999559044838, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;MISC&gt;, 25 month years-old , born in One Parkl...</td>\n",
       "      <td>[Patient AIQING, Beijing, The Johns Hopkins Ho...</td>\n",
       "      <td>[MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...</td>\n",
       "      <td>[0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 6, 7, 7, 8, 8]</td>\n",
       "      <td>MSW 100009632582 for his colonic polyps.</td>\n",
       "      <td>[[-0.4230000078678131, -0.027002999559044838, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;MISC&gt;, 25 month years-old , born in One Parkl...</td>\n",
       "      <td>[Patient AIQING, Beijing, The Johns Hopkins Ho...</td>\n",
       "      <td>[MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...</td>\n",
       "      <td>[0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 6, 7, 7, 8, 8]</td>\n",
       "      <td>He wants to know the results from them.</td>\n",
       "      <td>[[-0.4230000078678131, -0.027002999559044838, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;MISC&gt;, 25 month years-old , born in One Parkl...</td>\n",
       "      <td>[Patient AIQING, Beijing, The Johns Hopkins Ho...</td>\n",
       "      <td>[MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...</td>\n",
       "      <td>[0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 6, 7, 7, 8, 8]</td>\n",
       "      <td>He is not taking hydrochlorothiazide and is cu...</td>\n",
       "      <td>[[-0.4230000078678131, -0.027002999559044838, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;MISC&gt;, 25 month years-old , born in One Parkl...</td>\n",
       "      <td>[Patient AIQING, Beijing, The Johns Hopkins Ho...</td>\n",
       "      <td>[MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...</td>\n",
       "      <td>[0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 6, 7, 7, 8, 8]</td>\n",
       "      <td>He said he has cut his alcohol back to 6 pack ...</td>\n",
       "      <td>[[-0.4230000078678131, -0.027002999559044838, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;MISC&gt;, 25 month years-old , born in One Parkl...</td>\n",
       "      <td>[Patient AIQING, Beijing, The Johns Hopkins Ho...</td>\n",
       "      <td>[MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...</td>\n",
       "      <td>[0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 6, 7, 7, 8, 8]</td>\n",
       "      <td>He has cut back his cigarettes to one time per...</td>\n",
       "      <td>[[-0.4230000078678131, -0.027002999559044838, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;MISC&gt;, 25 month years-old , born in One Parkl...</td>\n",
       "      <td>[Patient AIQING, Beijing, The Johns Hopkins Ho...</td>\n",
       "      <td>[MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...</td>\n",
       "      <td>[0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 6, 7, 7, 8, 8]</td>\n",
       "      <td>P:</td>\n",
       "      <td>[[-0.4230000078678131, -0.027002999559044838, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;MISC&gt;, 25 month years-old , born in One Parkl...</td>\n",
       "      <td>[Patient AIQING, Beijing, The Johns Hopkins Ho...</td>\n",
       "      <td>[MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...</td>\n",
       "      <td>[0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 6, 7, 7, 8, 8]</td>\n",
       "      <td>Follow up with Dr. Hobbs in 3 months.</td>\n",
       "      <td>[[-0.4230000078678131, -0.027002999559044838, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;MISC&gt;, 25 month years-old , born in One Parkl...</td>\n",
       "      <td>[Patient AIQING, Beijing, The Johns Hopkins Ho...</td>\n",
       "      <td>[MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...</td>\n",
       "      <td>[0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...</td>\n",
       "      <td>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]</td>\n",
       "      <td>[0, 0, 0, 0, 1, 6, 7, 7, 8, 8]</td>\n",
       "      <td>Gilbert P. Perez, M.D.</td>\n",
       "      <td>[[-0.4230000078678131, -0.027002999559044838, ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-fa7c6047-8416-4aec-986d-4bac3a740721')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-fa7c6047-8416-4aec-986d-4bac3a740721 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-fa7c6047-8416-4aec-986d-4bac3a740721');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-cf6eb88b-c66d-446c-9c74-8d868a5d44c9\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-cf6eb88b-c66d-446c-9c74-8d868a5d44c9')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-cf6eb88b-c66d-446c-9c74-8d868a5d44c9 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                       de_identified  \\\n",
       "0  <MISC>, 25 month years-old , born in One Parkl...   \n",
       "0  <MISC>, 25 month years-old , born in One Parkl...   \n",
       "0  <MISC>, 25 month years-old , born in One Parkl...   \n",
       "0  <MISC>, 25 month years-old , born in One Parkl...   \n",
       "0  <MISC>, 25 month years-old , born in One Parkl...   \n",
       "0  <MISC>, 25 month years-old , born in One Parkl...   \n",
       "0  <MISC>, 25 month years-old , born in One Parkl...   \n",
       "0  <MISC>, 25 month years-old , born in One Parkl...   \n",
       "0  <MISC>, 25 month years-old , born in One Parkl...   \n",
       "\n",
       "                     entities_wikiner_glove_840B_300  \\\n",
       "0  [Patient AIQING, Beijing, The Johns Hopkins Ho...   \n",
       "0  [Patient AIQING, Beijing, The Johns Hopkins Ho...   \n",
       "0  [Patient AIQING, Beijing, The Johns Hopkins Ho...   \n",
       "0  [Patient AIQING, Beijing, The Johns Hopkins Ho...   \n",
       "0  [Patient AIQING, Beijing, The Johns Hopkins Ho...   \n",
       "0  [Patient AIQING, Beijing, The Johns Hopkins Ho...   \n",
       "0  [Patient AIQING, Beijing, The Johns Hopkins Ho...   \n",
       "0  [Patient AIQING, Beijing, The Johns Hopkins Ho...   \n",
       "0  [Patient AIQING, Beijing, The Johns Hopkins Ho...   \n",
       "\n",
       "               entities_wikiner_glove_840B_300_class  \\\n",
       "0  [MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...   \n",
       "0  [MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...   \n",
       "0  [MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...   \n",
       "0  [MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...   \n",
       "0  [MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...   \n",
       "0  [MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...   \n",
       "0  [MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...   \n",
       "0  [MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...   \n",
       "0  [MISC, LOC, ORG, LOC, LOC, MISC, PER, PER, PER...   \n",
       "\n",
       "          entities_wikiner_glove_840B_300_confidence  \\\n",
       "0  [0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...   \n",
       "0  [0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...   \n",
       "0  [0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...   \n",
       "0  [0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...   \n",
       "0  [0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...   \n",
       "0  [0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...   \n",
       "0  [0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...   \n",
       "0  [0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...   \n",
       "0  [0.5065, 0.8388, 0.30705, 0.4406, 0.40039998, ...   \n",
       "\n",
       "  entities_wikiner_glove_840B_300_origin_chunk  \\\n",
       "0               [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]   \n",
       "0               [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]   \n",
       "0               [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]   \n",
       "0               [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]   \n",
       "0               [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]   \n",
       "0               [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]   \n",
       "0               [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]   \n",
       "0               [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]   \n",
       "0               [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]   \n",
       "\n",
       "  entities_wikiner_glove_840B_300_origin_sentence  \\\n",
       "0                  [0, 0, 0, 0, 1, 6, 7, 7, 8, 8]   \n",
       "0                  [0, 0, 0, 0, 1, 6, 7, 7, 8, 8]   \n",
       "0                  [0, 0, 0, 0, 1, 6, 7, 7, 8, 8]   \n",
       "0                  [0, 0, 0, 0, 1, 6, 7, 7, 8, 8]   \n",
       "0                  [0, 0, 0, 0, 1, 6, 7, 7, 8, 8]   \n",
       "0                  [0, 0, 0, 0, 1, 6, 7, 7, 8, 8]   \n",
       "0                  [0, 0, 0, 0, 1, 6, 7, 7, 8, 8]   \n",
       "0                  [0, 0, 0, 0, 1, 6, 7, 7, 8, 8]   \n",
       "0                  [0, 0, 0, 0, 1, 6, 7, 7, 8, 8]   \n",
       "\n",
       "                                            sentence  \\\n",
       "0  Patient AIQING, 25 month years-old , born in B...   \n",
       "0           MSW 100009632582 for his colonic polyps.   \n",
       "0            He wants to know the results from them.   \n",
       "0  He is not taking hydrochlorothiazide and is cu...   \n",
       "0  He said he has cut his alcohol back to 6 pack ...   \n",
       "0  He has cut back his cigarettes to one time per...   \n",
       "0                                                 P:   \n",
       "0              Follow up with Dr. Hobbs in 3 months.   \n",
       "0                             Gilbert P. Perez, M.D.   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.4230000078678131, -0.027002999559044838, ...  \n",
       "0  [[-0.4230000078678131, -0.027002999559044838, ...  \n",
       "0  [[-0.4230000078678131, -0.027002999559044838, ...  \n",
       "0  [[-0.4230000078678131, -0.027002999559044838, ...  \n",
       "0  [[-0.4230000078678131, -0.027002999559044838, ...  \n",
       "0  [[-0.4230000078678131, -0.027002999559044838, ...  \n",
       "0  [[-0.4230000078678131, -0.027002999559044838, ...  \n",
       "0  [[-0.4230000078678131, -0.027002999559044838, ...  \n",
       "0  [[-0.4230000078678131, -0.027002999559044838, ...  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = '''Patient AIQING, 25 month years-old , born in Beijing, was transfered to the The Johns Hopkins Hospital.\n",
    " Phone number: (541) 754-3010. MSW 100009632582 for his colonic polyps. He wants to know the results from them.\n",
    "  He is not taking hydrochlorothiazide and is curious about his blood pressure. He said he has cut his alcohol back to 6 pack once a week.\n",
    "  He has cut back his cigarettes to one time per week. P:   Follow up with Dr. Hobbs in 3 months. Gilbert P. Perez, M.D.'''\n",
    "nlp.load(\"med_ner.jsl.wip.clinical en.de_identify.large\").predict(data,output_level = 'sentence')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xBDIzD5XyMGi"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
